Conventionally, image combining approaches have been known for dividing a subject into a plurality of partial images which are photographed, and combining the partial images with one another with estimation of a positional relationship among the respective partial images to generate an image called a mosaic image which has a wider viewing angle and a higher definition.
For example, Non-Patent Document 1 (Naoki Chiba, Hiroshi Kayano, Mitihiko Mino, Shouji Yasuda, “Image Mosaicking Based on Image Features,” Transaction of The Institute of Electronics, Information and Communication Engineers D-II, Vol. J82-D-II, No. 10, pp.1581-1589) discloses a method of generating a mosaic image by estimating a positional relationship among respective partial images based on motion vectors within the partial images generated by an image analysis. Specifically, sites (corresponding points) which are similar in distribution of luminance value are found from two partial images which were photographed at nearest times, and image conversion parameters are calculated for making a geometrical conversion (movement, rotation, scaling up/down) such that all of them overlap each other. Then, one partial image is used as a basis to geometrically convert the other partial image using the calculated image conversion parameters to generate a mosaic image.
The following model, which supposes that a subject is a flat surface, is often used for the image conversion parameters because of its simplicity.
                              (                                                                      u                  2                                                                                                      v                  2                                                                                    1                                              )                =                              (                                                            a                                                  b                                                  c                                                                              d                                                  e                                                  f                                                                              g                                                  h                                                  1                                                      )                    ⁢                      (                                                                                u                    1                                                                                                                    v                    1                                                                                                1                                                      )                                              [                  Equation          ⁢                                          ⁢          1                ]            where (u1, v1, 1), (u2, v2, 1) are homogeneous coordinates of respective corresponding points, a, b, c, d, e, f, g, h are image conversion parameters, and t is an arbitrary constant.
Also, for further simplifying the model, a restraint condition may be applied to the image conversion parameters, such as g=h=0, a=e=cos θ,−b=d=sin θ, and the like.
In the approach described in the aforementioned Non-Patent Document 1, a camera must be moved such that an overlapping portion exists between two partial images. Otherwise, no corresponding points would exist between the partial images, resulting in a failure in calculating the image conversion parameters. Also, if part of the subject is not photographed when the partial images are photographed, a mosaic image lacks the partial images, so that a user must exhaustively photograph the subject.
Patent Document 1 (Japanese Patent No. 2744265 (pages 5-7, and FIG. 3) describes an example of an image combining system for displaying a screen to guide a user having a camera to take a photograph such that the foregoing condition will be satisfied. The configuration of this conventional image combining system described in Patent Document 1 is illustrated in FIG. 1.
In FIG. 1, 101 designates a camera body; 102 a shutter button; 103 a display device, 104 and 105 a switch for instructing whether the camera is panned in the left-hand direction or right-hand direction; and 106 a lens for focusing a subject on a film or an imager device.
FIG. 2 illustrates an exemplary display by display device 103 of the image combining system illustrated in FIG. 1.
For example, when the camera is panned in the right-hand direction, as the right-hand direction is entered through switch 105, right side area 202 of partial image 201, which has been already photographed, is displayed in a left end area of displayed image 203. Further, left side area 205 of partial image 204 currently captured by the camera is combined in the remaining right side screen of displayed image 203 as a moving image, and the resulting image is displayed.
The user moves (pans) camera body 101 in parallel with an imaging plane, with reference to displayed image 203 on display device 103, such that right side area 202 of previously photographed partial image 201 is not discontinuous to left side area 205 of partial image 204 currently captured by the camera at the boundary therebetween, and depresses shutter button 102 to photograph the partial image. In this way, the mosaic image is generated by moving the camera toward a site of the subject not yet photographed, while including an overlapping area between the previously photographed partial image and the partial image currently captured by the camera, and sequentially photographing partial images.
However, in the conventional image combining system described in the above-mentioned Patent Document 1, the camera body must be moved in the vertical direction and horizontal direction, for example, if an image combining system is manufactured for photographing a subject which is broad in both the vertical and horizontal directions, based on the technique of Patent Document 1. Since a direction in which the camera is moved must be previously specified by the switch, a problem arises in that the operation is very complicated. Accordingly, the image combining system described in Patent Document 1 is of a configuration which is not suitable for photographing a subject which is broad in the vertical and horizontal directions.
Also, in the image combining system described in Patent Document 1, the camera body must be moved in parallel with the imaging plane, and the amount of movement must be adjusted such that a previously photographed partial image is not discontinuous to a partial image currently captured by the camera at the boundary therebetween. Thus, the user is required to perform subtle operations, causing a problem that the user experience considerable difficulty taking photographs.